PlaNet - Photo Geolocation with Convolutional Neural Networks

Is it possible to build a system to determine the location where a photo was taken using just its pixels? In general, the problem seems exceptionally difficult: it is trivial to construct situations where no location can be inferred. Yet images often contain informative cues such as landmarks, weather patterns, vegetation, road markings, and architectural details, which in combination may allow one to determine an approximate location and occasionally an exact location. Websites such as GeoGuessr and View from your Window suggest that humans are relatively good at integrating these cues to geolocate images, especially en-masse. In computer vision, the photo geolocation problem is usually approached using image retrieval methods. In contrast, we pose the problem as one of classification by subdividing the surface of the earth into thousands of multi-scale geographic cells, and train a deep network using millions of geotagged images. While previous approaches only recognize landmarks or perform approximate matching using global image descriptors, our model is able to use and integrate multiple visible cues. We show that the resulting model, called PlaNet, outperforms previous approaches and even attains superhuman levels of accuracy in some cases. Moreover, we extend our model to photo albums by combining it with a long short-term memory (LSTM) architecture. By learning to exploit temporal coherence to geolocate uncertain photos, we demonstrate that this model achieves a 50% performance improvement over the single-image model

From Multiview Image Curves to 3D Drawings

Reconstructing 3D scenes from multiple views has made impressive strides in recent years, chiefly by correlating isolated feature points, intensity patterns, or curvilinear structures. In the general setting - without controlled acquisition, abundant texture, curves and surfaces following specific models or limiting scene complexity - most methods produce unorganized point clouds, meshes, or voxel representations, with some exceptions producing unorganized clouds of 3D curve fragments. Ideally, many applications require structured representations of curves, surfaces and their spatial relationships. This paper presents a step in this direction by formulating an approach that combines 2D image curves into a collection of 3D curves, with topological connectivity between them represented as a 3D graph. This results in a 3D drawing, which is complementary to surface representations in the same sense as a 3D scaffold complements a tent taut over it. We evaluate our results against truth on synthetic and real datasets.Comment: Expanded ECCV 2016 version with tweaked figures and including an overview of the supplementary material available at multiview-3d-drawing.sourceforge.ne

MFGE8 does not influence chorio-retinal homeostasis or choroidal neovascularization in vivo

Purpose: Milk fat globule-epidermal growth factor-factor VIII (MFGE8) is necessary for diurnal outer segment phagocytosis and promotes VEGF-dependent neovascularization. The prevalence of two single nucleotide polymorphisms (SNP) in MFGE8 was studied in two exsudative or “wet” Age-related Macular Degeneration (AMD) groups and two corresponding control groups. We studied the effect of MFGE8 deficiency on retinal homeostasis with age and on choroidal neovascularization (CNV) in mice. Methods: The distribution of the SNP (rs4945 and rs1878326) of MFGE8 was analyzed in two groups of patients with “wet” AMD and their age-matched controls from Germany and France. MFGE8-expressing cells were identified in Mfge8+/− mice expressing ß-galactosidase. Aged Mfge8+/− and Mfge8−/− mice were studied by funduscopy, histology, electron microscopy, scanning electron microscopy of vascular corrosion casts of the choroid, and after laser-induced CNV. Results: rs1878326 was associated with AMD in the French and German group. The Mfge8 promoter is highly active in photoreceptors but not in retinal pigment epithelium cells. Mfge8−/− mice did not differ from controls in terms of fundus appearance, photoreceptor cell layers, choroidal architecture or laser-induced CNV. In contrast, the Bruch's membrane (BM) was slightly but significantly thicker in Mfge8−/− mice as compared to controls. Conclusions: Despite a reproducible minor increase of rs1878326 in AMD patients and a very modest increase in BM in Mfge8−/− mice, our data suggests that MFGE8 dysfunction does not play a critical role in the pathogenesis of AMD

Single-Image Depth Prediction Makes Feature Matching Easier

Good local features improve the robustness of many 3D re-localization and multi-view reconstruction pipelines. The problem is that viewing angle and distance severely impact the recognizability of a local feature. Attempts to improve appearance invariance by choosing better local feature points or by leveraging outside information, have come with pre-requisites that made some of them impractical. In this paper, we propose a surprisingly effective enhancement to local feature extraction, which improves matching. We show that CNN-based depths inferred from single RGB images are quite helpful, despite their flaws. They allow us to pre-warp images and rectify perspective distortions, to significantly enhance SIFT and BRISK features, enabling more good matches, even when cameras are looking at the same scene but in opposite directions.Comment: 14 pages, 7 figures, accepted for publication at the European conference on computer vision (ECCV) 202

Tissue Phenomics for prognostic biomarker discovery in low- and intermediate-risk prostate cancer

Tissue Phenomics is the discipline of mining tissue images to identify patterns that are related to clinical outcome providing potential prognostic and predictive value. This involves the discovery process from assay development, image analysis, and data mining to the final interpretation and validation of the findings. Importantly, this process is not linear but allows backward steps and optimization loops over multiple sub-processes. We provide a detailed description of the Tissue Phenomics methodology while exemplifying each step on the application of prostate cancer recurrence prediction. In particular, we automatically identified tissue-based biomarkers having significant prognostic value for low-and intermediate-risk prostate cancer patients (Gleason scores 6-7b) after radical prostatectomy. We found that promising phenes were related to CD8(+) and CD68(+) cells in the microenvironment of cancerous glands in combination with the local micro-vascularization. Recurrence prediction based on the selected phenes yielded accuracies up to 83% thereby clearly outperforming prediction based on the Gleason score. Moreover, we compared different machine learning algorithms to combine the most relevant phenes resulting in increased accuracies of 88% for tumor progression prediction. These findings will be of potential use for future prognostic tests for prostate cancer patients and provide a proof-of-principle of the Tissue Phenomics approach

Reanalysis in Earth System Science: Towards Terrestrial Ecosystem Reanalysis

A reanalysis is a physically consistent set of optimally merged simulated model states and historical observational data, using data assimilation. High computational costs for modelled processes and assimilation algorithms has led to Earth system specific reanalysis products for the atmosphere, the ocean and the land separately. Recent developments include the advanced uncertainty quantification and the generation of biogeochemical reanalysis for land and ocean. Here, we review atmospheric and oceanic reanalyses, and more in detail biogeochemical ocean and terrestrial reanalyses. In particular, we identify land surface, hydrologic and carbon cycle reanalyses which are nowadays produced in targeted projects for very specific purposes. Although a future joint reanalysis of land surface, hydrologic and carbon processes represents an analysis of important ecosystem variables, biotic ecosystem variables are assimilated only to a very limited extent. Continuous data sets of ecosystem variables are needed to explore biotic-abiotic interactions and the response of ecosystems to global change. Based on the review of existing achievements, we identify five major steps required to develop terrestrial ecosystem reanalysis to deliver continuous data streams on ecosystem dynamics

Wide-area mapping of small-scale features in agricultural landscapes using airborne remote sensing

Natural and semi-natural habitats in agricultural landscapes are likely to come under increasing pressure with the global population set to exceed 9 billion by 2050. These non-cropped habitats are primarily made up of trees, hedgerows and grassy margins and their amount, quality and spatial configuration can have strong implications for the delivery and sustainability of various ecosystem services. In this study high spatial resolution (0.5 m) colour infrared aerial photography (CIR) was used in object based image analysis for the classification of non-cropped habitat in a 10,029 ha area of southeast England. Three classification scenarios were devised using 4 and 9 class scenarios. The machine learning algorithm Random Forest (RF) was used to reduce the number of variables used for each classification scenario by 25.5 % ± 2.7%. Proportion of votes from the 4 class hierarchy was made available to the 9 class scenarios and where the highest ranked variables in all cases. This approach allowed for misclassified parent objects to be correctly classified at a lower level. A single object hierarchy with 4 class proportion of votes produced the best result (kappa 0.909). Validation of the optimum training sample size in RF showed no significant difference between mean internal out-of-bag error and external validation. As an example of the utility of this data, we assessed habitat suitability for a declining farmland bird, the yellowhammer (Emberiza citronella), which requires hedgerows associated with grassy margins. We found that ∼22% of hedgerows were within 200 m of margins with an area >183.31 m2. The results from this analysis can form a key information source at the environmental and policy level in landscape optimisation for food production and ecosystem service sustainability

Speciation-controlled incipient wetness impregnation: A rational synthetic approach to prepare sub-nanosized and highly active ceria–zirconia supported gold catalysts

On the basis of calculated thermodynamic species distribution diagrams and by appropriately controlling the pH of aqueous HAuCl4 solutions, it has been possible to prepare, using a Speciation-controlled Incipient Wetness Impregnation (ScIWI) approach, Au catalysts supported on ceria–zirconia mixed oxides featuring both high gold loadings and excellent metal dispersions. This rational synthesis method is carried out at room temperature. It is both much simpler, in equipment terms, and less expensive than widely used Deposition–Precipitation (DP). Moreover, the use of ScIWI allows overcoming the severe limitations of previously assayed impregnation methods. With this procedure it is possible to prepare active catalysts in CO oxidation with high efficiency in terms of gold precursor usage, i.e. minimizing Au losses during synthesis. Therefore this, quite amenable, novel strategy for the facile preparation of highly dispersed supported gold catalysts gathers the necessary requirements for both its use at lab scale and an easy scaling-up to industrial levels